This application claims priority from Japanese patent application number 11-365016, filed Dec. 22, 1999, which is hereby incorporated herein by reference in its entirety.
The present invention relates to a method and an apparatus for increasing, using a rational number multiplier, the resolution of an image consisting of a set of pixels that are represented by digital values.
Most conventional image expansion techniques employ a natural image, such as a scenery picture, as an expansion target, and substantially there are no stepped edges, such as thin lines, in an image. This is mainly because information is recorded, while the stepped edge is transformed, by the low-pass effects of an image pickup apparatus, into a form represented by, for example, a sigmoid function (a differentiable, continuous function, f(x)=1/(1+(exe2x88x92x)). Therefore, conventionally, improvements have been performed to determine how to make the outline, which tends to be more ambiguous than necessary, to appear natural, with the assumption that the original image was obtained by sampling a low-pass filtered image.
For resolution conversions, which require the least computations and can be easily implemented, the replica and nearest neighbor methods are used. With the replica method, the simplest, an image is expanded (n+1)/n times by copying the same pixel at every n-th pixel. And with the nearest neighbor method, an image is expanded by copying the pixel of the original image that is closest to the coordinates obtained by converting the resolution. Both the replica method and the nearest neighbor method provide substantially the same results; that is, the mixing of colors among pixels does not occur, and changes in tones are persistent.
Also, a bi-linear interpolation method has been established for the conversion of resolutions. With this method, the coordinates of a pixel point for a resultant image are inversely projected onto the coordinates of the original image. Then, the pixels (four surrounding points, two points on each of two sides, or only one point located at the identical coordinates) of the original image in the vicinity of the coordinates are weighted, using distance, to obtain an average value, which subsequently is regarded as the color of the resultant image.
Furthermore, a bi-cubic interpolation method is also well known. This method applies the same principle as does the bi-linear method, but for two surroundings in the vicinity (16 points). According to this method, it is assumed that the differential continuity and the value change (gradient) of the original image are sufficiently moderate. Colors, which are enhanced by the weighing of parameters, are more clearly defined than they are with the bi-linear method.
In addition, there is a multirate system that is obtained by generalizing the preceding interpolation methods and that requires more computations. The basic configuration of the multirate system provides for low-pass filtering to be performed after up-sampling using zero-value interpolation, and down-sampling is performed to expand an image at a predetermined rate. This configuration can theoretically include the bi-linear method and the bi-cubic method by adequately selecting the low-pass filter. Actually, in many cases a poly-phase configuration or a filter bank configuration is mounted in order to increase the computational efficiency.
However, according to the replica method and the nearest neighbor method, the pixel of an original image is simply held at a new spatial sampling point, and the expanded line width differs depending on the coordinate positions. Since the sensitivity to frequency components of human eyes is high to a low frequency for the angle unit, a serious problem occurs in the readability of a line width in these methods. The study of an eye model is described in reference IEEE Trans. on Image Processing, vol. 4, pp. 1460-1464, 1995. According to this reference, when an oblique line is expanded, it can be assumed that the distortion in a low frequency band, such as an increase in the width of jaggies, can be easily perceived and that the quality will be deteriorated.
Further, according to the bi-linear interpolation method, when the coordinates that are determined for an inversely projected image are located at the same distance on either side of a one-dot width line, the resultant image will invariably represent a line as wide as two lines of a half a color. Therefore, problems occur relative to the uniformity of colors, the legibility of characters and the reproduction fidelity of colors. Although an image in a photo may appear to be satisfactory, the overall impression is of a blur.
Furthermore, the bi-cubic interpolation method has the same problem as does bi-linear interpolation method, where one line is changed into two lines half a color. Thus, the accurate reproductivity of colors of the screen image of a personal computer (hereinafter referred to as a PC) is still problematic. Also, there is a slight occurrence of ringing at sharp boundaries between middle tone colors.
Moreover, when an arbitrary low-pass filter is employed for the multirate system, checkerboard distortion may appear in a resultant image, thereby indicating that the filter designs that can be used are limited. That is, limitations are placed on filter designs in accordance with the images that are to be provided, and required filter conditions, such as the one according to which the filter property for a passing band and a blocking band must be flat, or a condition according to which a down sampling filter is separately provided to introduce an average operation. It is preferable that an FIR (Finite Impulse Response) filter be employed to maintain the linear phase of an image; however, usually, a higher order filter is required to obtain an image quality that is higher than that acquired by the bi-linear or bi-cubic interpolation method.
Conventionally, many improved methods have been proposed to correct outlines produced by bi-linear or bi-cubic interpolation that are blurred or stair-step shaped. However, for these methods it is premised that a boundary has at the least a specified size, and thus a font that has a structure consisting of one pixel width can not be satisfactorily coped with.
In reference Proceedings of the 1999 IEICE General Conference D-11-120, a resolution conversion method employing the re-formation of an oblique outline is proposed whereby the edge of an original image is detected, and an edge having an inclination that corresponds to that of the detected edge is re-formed for the resultant image, so that jaggies that are generated due to the expansion of an oblique line can be replaced on the resultant image with fine steps. According to this method, however, a 3xc3x973 sized filter, at the least, must be employed to detect the edge, and the fine structure of a font can not be expanded.
In reference IEEE Proceedings of International Conference on Image Processing 1997, vol. I, pp. 267-270, October 1997, a method is described whereby linear expansion is employed for an edge that is enhanced in an original image. However, this method is not effective when a structure is smaller than the size of an edge detection filter, and outstanding stair-stepping jaggies are produced unless shaping is performed for an inclined boundary.
Further, in reference IEEE Transactions on Image Processing, vol. 8, no. 5, pp. 629-639, May 1999, a method is proposed for changing a distance weighing that is used for the interpolation of an outline. According to this method, a steep change on the original image is detected by four pixel points that are on both sides of a target pixel, and the distance from the interpolation point that is used for bi-linear or bi-cubic interpolation is changed. While this method is effective for obtaining a clear outline for a natural image, however, such as for a PC screen image, substantially it is only as effective as the nearest neighbor method.
To resolve the above technical problems, it is one object of the present invention to use an outstanding color to represent a font or a thin line on a graphics screen that is handled by an information display device, such as an LCD (Liquid Crystal Display) panel or a projector, and that includes many stepped edges, such as thin lines.
It is another object of the present invention to provide a new image resolution conversion method for enabling real time processing even for a large number of pixels.
To achieve the above objects, according to the present invention, an image conversion method for converting original input image data into image data that is expanded less than twice, comprises the steps of: examining a connection between a target pixel in the original input image data and pixels neighboring the target pixel; employing the connection to determine whether the target pixel constitutes a one-pixel width line; determining, when the target pixel constitutes a one-pixel width line, the value of a resultant pixel in the expansion image data in order to retain the one-pixel width even in the expansion image data.
For the determination made to decide whether a target pixel constitutes a one-pixel width line, if the pattern of the connection with the target pixel is a vertical, horizontal or oblique one-pixel width line, it can be ascertained that the pertinent pixel constitutes a one-pixel width line. The value of the resultant pixel is obtained as the digital output for an R (red), G (green) or B (blue) color signal. It should be noted that the color signals are not always limited to R, G and B, and that the value of the resultant pixel may also be output as the value of a color signal for X, Y or Z, L*a*b*, Y, M, C or K.
The value of a pixel is determined, so that the connection with the neighboring pixels is retained when the value of the resultant pixel in the expansion image data is determined. For example, for a pattern wherein lines obliquely intersect, the connection of lines that intersect each other can be maintained.
The original image data are color image data. And when between the target pixel and the neighboring pixels colors match or are similar, it is ascertained that the connection with the neighboring pixels is established. The value of the resultant pixel in the expansion image data is determined so that the color of the resultant pixel matches the color of the target pixel. This arrangement is preferable because a fine structure can be prevented from being battered due to a feathering phenomenon, and because clear, expanded character images, such as fonts, can be obtained. To facilitate the determination as to whether colors are similar or not, a differential value, for example, for a color that can be regarded as being substantially the same as the color of the target pixel is stored in memory in advance. Then, when a color value falls within the range represented by the differential value in the memory, it is ascertained that the color is similar. Further, the present invention can also be provided so that when the color of neighboring pixels matches the color of the target pixel, between the original input image data and the expansion image data the number of colors is substantially unchanged.
According to the present invention, an image conversion method for converting original input image data into expansion image data that is expanded a rational number, which is not an integer, of times, comprises the steps of: detecting a connection between a target pixel in the original input image data and pixels neighboring the target pixel; employing a relationship between a coordinate position for a pixel in the original image data and a coordinate position for a resultant pixel in the expansion image data to define fixed mapping that is prepared for a plurality of types into which the relationship is sorted; and determining a value for the resultant pixel based on the mapping, while the connection with the neighboring pixels is also maintained for the expansion image data.
The increase in the size of the expansion image data provided by conversion is less than twice, and the mapping is defined by a combination of a coordinate point type that does not perform expansion and a coordinate point type that basically performs expansion. There are four coordinate point types: a type for dominating one peripheral pixel of the resultant pixel, a type for vertically dominating two pixels, a type for horizontally dominating two pixels, and a type for dominating four neighboring pixels. The sorting into these coordinate point types can be used for a rule-based expansion in the invention. Further, in the above example, the coordinate point type that does not perform expansion may be the type that dominates one neighboring pixel of the resultant pixel, while the coordinate point type that basically performs expansion may be the type that vertically dominates two pixels, that horizontally dominates two pixels, or that dominates four pixels of the resultant pixel.
When the connection with the neighboring pixels is employed to determine that the target pixel constitutes a one-pixel width line, a one-pixel width is maintained for the coordinate point type that basically performs expansion, and the value of the target pixel is allocated for a specific resultant pixel, while the value of a peripheral pixel of the target pixel is allocated for a resultant pixel other than the specific resultant pixel.
Further, according to the present invention, an image processing apparatus for producing, by conversion, expansion image data that is less than twice the size of the original input image data that is converted, comprises: determination means, for employing a connection of a target pixel in the original image data with peripheral pixels of the target pixel to determine whether the target pixel constitutes a one-pixel width line; rule application means, for employing a relationship between a coordinate position of a pixel in the original image data and a coordinate position of a resultant pixel in the expansion image data to define a plurality of types into which the relationship is sorted, and for applying a rule to the plurality of types that are defined; and output means, for, when the determination means determines that the target pixel constitutes a one-pixel width line, outputting a value for a resultant pixel that in accordance with a specific rule for maintaining a one-pixel width line applied by the rule application means constitutes the expansion image data. The determination means, the rule application means and the output means can be provided by mounting an algorithm as software, or can be implemented by hardware incorporating a circuit.
The determination means employs the connection with the peripheral pixel to determine whether the target area at the least forms a two-pixel width area. And when it is ascertained by the determination means that the target area at the least forms a two-pixel width area, the output means outputs the value of the resultant pixel to increase the width of the area.
When, like a striped pattern, a line that is to be expanded can not be determined locally, from the lines in the pattern one line can be selected that connects to a portion having a predetermined size in an area that may be expanded vertically and horizontally.
According to the present invention, an image processing apparatus for converting original input color image data having a low resolution into expansion color image data having a high resolution, comprises: width examination means, for comparing a target pixel in the original color image data with image data that are produced by peripheral pixels to determine whether the target pixel is a pixel having a small width or a large width; and determination means, for determining a value for a resultant pixel that constitutes the expansion color image data, so that a small width is maintained when the width examination means ascertains that the target pixel has the small width, or so that a large width is further expanded when the width examination means ascertains that the target pixel has the large width. This arrangement is preferable because a thin line having a clear, sharp image can be maintained, and an expanded font image is much easier to see.
In addition, the expansion color image data is obtained by increasing less than twice the resolution of the original color image data. The width examination means then determines whether a pixel having a small width is a pixel that constitutes a one-pixel width line, and the determination means determines the value of the resultant pixel, so that the one-pixel width is maintained for a corresponding line of the expansion color image data.
The image processing apparatus further comprises: the color examination means, for examining the color of the target pixel in the original color image data, and the determination means, for determining the value of the resultant pixel by using unchanged, without mixing with the color that is identified by the color examination means, the color of the target pixel. This arrangement is superior because a sharp, expanded image can be obtained wherein the feathering of color is reduced.
According to the present invention, an image processing apparatus, which produces, by conversion, high resolution expansion image data that is less than twice the size of the original input image data that is converted, and which outputs the expansion image data, comprises: a detector for detecting a connection of a target pixel in the original image data in accordance with a value of the target pixel and a value of neighboring pixels; a width expansion determiner for employing the connection that is detected by the detector to determine whether the width of the target pixel can be expanded; and a rule application unit for employing a relationship between a coordinate position of a pixel in the original image data and a coordinate position of a resultant pixel in the expansion image data to define a plurality of types into which the relationship is sorted, and for employing the output from the width expansion determiner to apply a specific rule for the plurality of types that are defined.
Furthermore, the width expansion determiner employs the connection obtained by the detector to determine whether the target pixel is a portion, having at the least a specific size, in an area that can be expanded vertically and horizontally. When the target area is the portion, it is ascertained that the width of the target pixel can be expanded. This arrangement is preferable because even for an image, such as a stripe pattern, for which whether expansion is possible can not be determined merely from the pixel width, the width can be expanded while taking into the account the continuity of the target pixel and the above described portion.
Moreover, according to the present invention, an image processing apparatus, which produces, by conversion, expansion image data that is less than twice the size of the original image data that is converted, comprises: input means, for receiving the original image data; connection examination means, for examining the connection of a target pixel in the original image data input by the input means and neighboring pixels; judging means, for employing the connection obtained by the connection examination means to determine whether the target pixel constitutes a one-pixel width line; and determination means, for, when the judging means determines that the target pixel constitutes a one-pixel width line, determining a value for a resultant pixel in the expansion image data, so that the one-pixel width in the expansion image data is maintained.
The value of a pixel is determined, so that the connection with the neighboring pixels is retained when the value of the resultant pixel in the expansion image data is determined.
Further, the original image data input by the input means are color image data. And when between the target pixel and the neighboring pixels colors match or are similar, the connection examination means ascertains that the connection with the neighboring pixels is established. The determination means determines the value of the resultant pixel in the expansion image data so that the color of the resultant pixel matches the color of the target pixel.
The above described processing can be performed for the determination of whether the target pixel constitutes a one-pixel width line, the examination of the connection and the determination of whether the colors are similar using previously mentioned ways, so no further explanation for these processes will be given.
According to the present invention, an image display apparatus, which produces, by conversion, high resolution expansion image data that is less than twice the size of the original low resolution image data that is converted, comprises: an original image data input unit for receiving the original low resolution image data; a connection detector for detecting a connection of a target pixel in the original image data input by the original image data input unit and neighboring pixels; a rule application unit for employing a relationship between a coordinate position of a pixel in the original image data and a coordinate position of a resultant pixel in the expansion image data to define a plurality of types into which the relationship can be sorted, and for applying a specific rule for the plurality of types that are defined; a results output unit for determining the value of the resultant pixel in accordance with the rule applied by the rule application unit, and for outputting the expansion image data; and an image display unit for displaying an expanded high resolution image based on the output of the results output unit, wherein, when the connection detector ascertains that the target pixel constitutes a one-pixel width line, the rule application unit applies a rule that will maintain the one-pixel width even in the expansion image data.
The image display unit corresponds, for example, to a display panel, such as an LCD panel, or a projector. When the connection detector ascertains that the target pixel is connected to the neighboring pixels, the rule application unit applies the rule to maintain the connection with the expansion image data. This arrangement is superior in that the rule can be formulated even when the foreground and the background are not identified.